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Abstract — Interactive 3D visualization of human organs is 
becoming essential in medical field for aiding surgeons’ 
knowledge of complex anatomies. Recent researches highly 
focus on the development of virtual environments since they are 
programmable, flexible and provide realistic experience. Virtual 
reality serves as a tool for interaction between the surgeon and 
the patient model. This helps the surgeons to plan the surgeries 
in an efficient way irrespective of its complexity. Trauma 
condition refers to serious life threatening injuries that cause 
psychological shock. Virtual reality based traumatic surgery 
simulation provides hands-on experience for surgeons. This 
paper reviews the state-of-the-art techniques of virtual reality in 
surgery simulation and trauma care. 

Index Terms: surgery planning, surgery simulation, surgery 
training, trauma care, virtual reality 

I. INTRODUCTION 

Virtual Reality (VR) is an experience, an interface between 
human and computer generated environment in a naturalistic 
way. Varied applications of VR are in medical surgery 
simulation, neuropsychological assessment, rehabilitation, 
assistance for accident trauma care, telesurgery, military, 
education, architectural design evaluation, gaming, 
entertainment, rapid prototyping etc. Visualization of 
complex medical data is mandatory for better planning and 
simulation of surgeries. VR for surgical planning has gained 
wide acceptance throughout. It provides surgeons and 
physicians with medical avatars - real like body parts that can 
interact with external devices like surgical instruments [1]. 
This has created revolution in developing surgery techniques 
with the primary goal of minimizing patient’s suffering, time 
and complexity of operation. 

Minimally Invasive Surgery (MIS) [2] is the practice of 
operating the patients with minimal damage, reduced post- 
operative pain and minimal hospital stay. It is a technique 
which requires a lot of training for the surgeons before 
operating because of its minimal vision, minimal mobility of 
surgical instruments, complexity and perfect synchronization 
of hand and eye. However the recent 
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developments in surgery simulation systems have made 
teaching and performing surgery an easier task even for 
interns. 

MIS (Laparoscopy) has increasingly been practiced for 
surgery in orthopedics, otolaryngology, abdominal surgery, 
gynecology, gastroenterology etc. In neurosurgery, an 
interactive simulator, Dextroscope, is being actively involved 
to improve the planning processes. Cyberscalpel is a surgical 
planning tool created by NASA researchers, for treating the 
astronauts when they are in space. ProMIS, a mixed reality 
simulator system for laparoscopic surgery comprises software 
and a number of models for training. The movement of hand, 
position, velocity and direction of surgical instruments can be 
tracked. 

AYR A [3] is an efficient tool to plan surgical simulation. It 
contains all surgical tools and runs on conventional PCs. Here 
the 2D radiological images can be viewed as an interactive 
3D model with proper preprocessing and segmentation. This 
model is prototyped to get a physical biomodel from an 
industrial manufacturing environment. Such models are used 
in reconstruction surgery like maxillofacial surgery, 
craniofacial surgery, orthopedics, etc. 

Road accidents are increasing alarmingly in all parts of the 
world. According to the Ministry of Road Transport and 
Highways - Transport Research wing, New Delhi, India, 
around 5 lakh accidents had occurred in the year 2011 which 
keeps on increasing every year. This implies the necessity of 
post accident trauma care throughout the country. Accident is 
a crucial situation where the extent of injuries and the 
availability of expert physicians are unknown. Apart from 
these, the injured person must be given proper treatment 
without any delay. This necessitates trauma care units. 

A virtual environment which approximates working on real 
patients serve as a tool to better train the surgeons evaluate 
and treat the patients in emergencies. Simulation of a wide 
variety of accident cases in VR will enhance the performance 
of the physicians and interns in trauma surgery. 

This paper reviews some important virtual reality based 
simulation methods for planning and training in liver, 
vascular, hip joint, urethral, dental, orthognatic cardiac and 
abdominal trauma surgeries. Section II of this paper 
discusses the role of VR in surgery simulation and planning 
and Section III explains the role of VR in post accident 
trauma surgeries. 
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II. ROLE OF VR IN SURGERY SIMULATION AND 

PLANNING 

A. VR application in Liver Surgery 

Liver is one of the largest organs of the human body, 
located below the right lung. It is divided into two lobes - 
right and left lobes. Liver cancer consists of malignant hepatic 
tumors in or on the liver. The main causes cited for it are 

• chronic hepatitis B and C infection. 

• cirrhosis of the liver. 

• diabetes mellitus. 

• exposure to toxins. 

• smoking. 

The only cure for liver cancer is liver resection, Hepatectomy 
- removal of diseased tissues. Liver resection strategies are 
mainly classified into anatomical resections and atypical 
resections. Anatomical resections are removal of liver 
segments and are preferred when the tumor is inside the liver 
.Atypical resection is preferred when the tumor is in the 
peripheral section of the liver. Preoperative planning helps to 
make appropriate decision of the resection strategies. 

[4] and [5] have proposed tools for liver surgery planning. 
Key indices influencing surgery are tumor location and size 
and postoperative liver function. Hence, Preoperative 
planning plays a vital role in Hepatectomy. In general, 
surgeons rely on CT images for this purpose. They mentally 
visualize the 3D model of the organ from the stack 2D slices. 
Due to the anatomical variations; this kind of visualizations 
may lead to misinterpretations. An Augmented Virtual (AR) 
reality scheme is used to plan the hepatic surgery. This 
scheme [4] begins with the 3D modeling of the 2D CT liver 
image of the patient. As an initial step, all the functional 
organs in the abdomen are modeled to 3D. The organs other 
than liver (heart, lungs, spleen, kidneys) are cropped out using 
appropriate thresholding and gray level histogram techniques 
as shown in Fig 1(a). The patient liver is then delineated by 
the morphological deforming of the liver template (Fig 1(b)) 
which is embedded into the 2D imaging technique. With this 
liver model, the segmentation is carried out to categorize the 
voxels as - hepatic tissue, lesions and vessels. This 
classification is further refined using topological and 
geometrical constraints to automatically delineate the hepatic 
vein and the portal vein. 



Fig. 1. (a) Anatomical structure delineation (b) Patient liver 

segments 

In [5], Nicolau et al. developed a tool, a prototypal 
augmented reality guiding abdominal surgery. This tool 
allows simulating any kind of coelioscopic surgeries and 
provides a realization of virtual resections. With the ribs as 
the landmark, the 3D visualization of the liver is 
superimposed on the real operative video of the patient via an 


optimal registration model. More commonly, the Extended 
Projected Point Criteria (EPPC) [6] is used for the 
registration purpose. Thus this system makes the preoperative 
2D data more informative and proves to be highly efficient. 

The center for Medical Diagnosis systems and 
visualization, Germany in the early 90 ’s presented two 
desktop based systems, Hepavision and Surgery planner for 
planning liver transplantations and resection. This used the 
portal vein tree and hepatic vein structure to generate the 
resection. Visualization and adjustment of safety margins is 
possible. User interaction however is limited. In 2006 [7] 
developed a tool, Liver Planner to help physicians find the 
best resection plan for individual patients. This is an 
immersive tool with the following hardware components: a 
stereoscopic large screen projection system, a Tablet PC, an 
eye of Ra input device, and shutter glasses. It has three main 
phases: image analysis stage, segmentation refinement stage 
and treatment planning stage. Image analysis is segmentation 
of tissues and vessels. In tissue segmentation, segmentation of 
liver and tumor has been performed by an extended multi 
object fuzzy connectedness. For segmentation of the 
diaphragm dome surface a 3D active appearance model 
(AAM) is used and for segmenting heart from liver, AAM 
matching algorithm is used. A model based vessel mining 
approach based on tube detection filter and the nearest 
neighbor approximation is used to segment vessels. Fig. 2 
shows the anatomical resection simulation. 



(a) (b) (c) 

Fig. 2. (a) Automatic selection and removal of segments of 
liver with a tracking pencil (b) user draws contour on Tablet 
PC (c) segmentation refinement results 


The refinement stage is an interactive stage which corrects for 
errors in the segmented regions. Direct deformation tools for 
correcting small defects and template shape tool for 
correcting large defects are made available in the Liver 
Planner. The treatment planning stage includes interactive 
branch labeling, liver segment approximation, safety margin 
calculation and spatial analysis tools. Compared to a 3D 
desktop based planning system, hardware cost is the most 
crucial factor in a Liver Planner. 

B. VR application in Vascular Surgery 

The surgical procedures followed by surgeons need a lot 
of training and practice. The advances in virtual reality 
systems aid doctors in the pre operative training. [8] 
describes a virtual reality toolkit for minimal invasive 
vascular surgery. The software framework is constituted of 
the following five modules: 

• graphics rendering engine - provides 3D graphics 
application programming interface and interactive 
user interface 
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• physics engine - provides collision detection and soft 
body dynamics 

• interface module - includes load .bullet file module 
and GUI module 

• math module - in charge of computing the force 
simulation and the force feedback 

• core module - in charge of providing useful APIs for 
applications especially for basic soft body dynamics 
and simulation 

Depth buffer and night vision techniques are incorporated to 
the software to provide more effective virtual reality 
simulation as in Fig 3. 



Fig. 3. A 3D simulation of the X-ray vascular structure 

The toolkit also encompasses soft body dynamics, to simulate 
the motion and properties of deformable objects. The 
simulation can be obtained by using the following 
approaches: mass-spring model, rigid-body based 

deformation and energy minimization methods. This software 
toolkit enables users to create more views on multiple screens 
and more graphics contexts on one view as in Fig 4. 



Fig. 4. User interface 

In order to provide effective interaction between the surgeons 
and the toolkit, the simulator is controlled by keyboard 
interactive mode and haptic device mode. The robotic guide 
wire operating system serves as the haptic device to provide 
the real effect of VR. 

C. VR application in Hip joint surgery 

Structural abnormality is cited to be the main cause of hip 
joint disease. Osteotomy is the remedial measure performed 
to remove the necrotic part of the femoral head from the 
bearing section of the acetabulum in order to replace with the 
healthy part of the femoral head [9]. However the preoperative 
planning for this surgery is done using the Computed 
Tomography (CT) images according to the experience of 
doctor. By using the virtual reality system, efficient 
preoperative strategies can be planned. This VR system has 
two modules: the data preparation and the mechanical 
analysis module. The data preparation module produces a 
three dimensional image (Fig 5) of the hip joint using Visual 
Studio.Net and visualization toolkit and the mechanical 
analysis is realized using Quest 3D. On an assumption that 
the joint surface consists of numerous small triangular planes, 
the authors have formulated a rigid- body spring model to 


help analyze the pressure distribution on the hip joint. Thus 
the results of osteotomy help in increasing the success rate of 
surgery. 



Fig. 5. 3D model of Hip joint 

D. VR application in urethral surgery 

Yet another application [10] where VR surgery simulation 
systems are applied is for training Transurethral Resection of 
the Prostate (TURP). The prostate gland is located next to the 
bladder in human males. Benign enlargement of the prostate 
known as Benign Prostatic Hyperplasia (BPH), results in 
blockage of urine. This creates the need for removing the 
inner lobes in order to relieve the obstruction. 



prostate capsule 



— 3 " 

ultrasound probe 


verumontanum 


a 

anum j.s 


adenoma bladder 
neck 


(c) 

Fig. 6. a) device that emulates a real resectoscope b) Graphic 
prostrate model and resecting loop c) Transurethral 
ultrasound imaging of prostrate 


The simulator includes a 3D deformable tissue model of the 
prostate (Fig 6b) from the ultrasound images [10] by a 
modeling software to simulate resection of the tissue. 
Realistic simulation of the movements of the surgeon has 
been obtained through a disk ring array which consists of an 
embedded electronic system (Fig 6a). This consists of five 
microcontrollers for movement monitoring and these 
movements reflect the interaction between surgeon and tissue 
model. 


E. VR application in dental surgery 

Realistic haptic devices and graphical displays are the 
recent training systems used for dental simulation. These 
haptic devices can simulate cutting, dental implants, 
exploration etc. A light weighted prototype haptic device, 
iFeel designed by Li, Chang and Wang [11] outputs a 3D 
force similar to the force generated in real operation. The 
performance characteristics of the haptic device are decided 
based on the Degree of freedom (Dof), maximum output 
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power, workspace, stiffness and resolution. Dof is the 
freedom to move freely in the oral cavity which is 6 degree. 
Maximum output power is the maximum exertable force on 
the device which is designed as 5N. Workspace is both 
translational and rotational workspace and the posture tool 
was created in the virtual tool named Phantom Omni. The 
size and posture of the virtual oral cavity are similar to the 
real tool (Fig. 6a) and the tools posture in virtual surgery can 
be recorded. The simulated device has maximum stiffness 
criteria of 14.5 (N/mm). All the above said characteristics are 
analyzed quantitatively and iFeel3 as a dental mirror is shown 
in Fig. 7b. 


(a) 

Fig. 7. (a) Virtual dental environment (b) iFeel3 used as a 

dental mirror 

Another paper on dental simulation [12] deals with haptic 
sensation features on tooth grinding and preparation. Tooth 
grinding is shaping the tooth for further treatment. Grinding 
operation is by high speed rotating burs. The force applied for 
this operation should neither be too high nor too low. Higher 
force might damage the tooth tissues and lower force might 
cause prolonged painful treatment. That is why, appropriate 
haptic sensation features has become important for planning 
operations. In using haptic systems, the fidelity of the 
feedback force and stability of the mechatronic device are the 
most important things to be taken care of. A simplified 
feedback force based on grinding force model has been 
developed in [12]. The feedback force is generated by the 
grinding interaction between the tooth and the dental bur (Fig. 
8a) on the hand piece. The factors that affect the grinding 
force are the shape of the bur, behavior of the hand piece and 
different tooth sections and environment. In dental surgery, 
the tooth is static and the grinding tool undergoes rotational 
and translational velocities. The different forces acting are 
calculated using virtual coupling to retain stability. A voxel- 
based grinding simulator has also been developed to 
overcome the force discontinuity and the feedback force 
generated is shown to be consistent with the real grinding. 
These methods are implemented in Visual Studio 2005 with 
Open Haptics and OpenGL and have been integrated into a 
dental training system. These simulation platforms have been 
effective in creating better alternative for traditional training 
with plastic teeth or removed teeth from patients. 


(a) (b) 

Fig. 8. (a) Bur in different shapes (b) prepared cavity 


F. VR application in orthognatic surgery 

The common task of plastic surgeons is to correct the 
dislocated bone fragments up to the complete modeling of 
facial regions. In these cases, individual anatomy and 
physiology sounds to be important. [13] presented a computer 
assisted modeling, planning and simulation system to aid 
surgeons to effectively perform the surgical corrections. This 
allows the surgeons to first work on a 3D virtual model of the 
patient before actual surgery. The most important factor here 
is the anatomy and physiology of the patient's skull before 
any disproportions. This is accomplished by the studies on 
facial proportions and characterization of faces along with the 
database of healthy morphological anatomies. The 3D model 
of the deformed skull is obtained by the projection of 2D X- 
ray images as shown in Fig 9. 



Fig. 9. 2D planning in orthognatic surgery using x-ray image 

In addition to the 3D reconstruction, the changes in the soft 
tissues due to the bone dislocation and teeth location must 
also be focused. The 3D model of the tissue boundaries are 
obtained from a sequence of tomographic images with sub 
voxel accuracy resulting in a surface model with millions of 
triangles. Such a model should not only represent the 
anatomically and topologically correct tissue surfaces, but 
also the mixed tissue volumes. The actual dislocations can be 
determined by the measuring the distances and angles 
between the characteristic points. This is referred to as the 
cephalometric analysis. The characteristic points can be fixed 
either in the image data or on the skull and skin surface 
respectively. For a standard 3D cephalometric analysis, the 
patient model is aligned with the sagitttal median and the 
Frankfort Horizontal plane. Comparison of the individual 
skull shape with a statistical reference, would generate 
proposals for reconstruction. As the cephalometric analysis 
provides the deviation of the facial bones from the symmetry 
and the functional analysis provides the functional 
impairment, osteotomy is the way to treat the dysplasia. This 
requires thorough planning. 

The computer generated osteotomy has its base on draw- 
and-cut principle [14]. The initial step is to draw the 
osteotomy lines and then to compute path to the surfaces to be 
cut. The transection of nerves, vessels and underlying teeth 
roots can be revealed by mapping the cut surfaces and the 
tomographic images. The mobilized parts of the model are 
made to be rotated and translated with single degree of 
freedom over a rotational axis. Another aspect to be taken 
care of in osteotomy is the collision detection and avoidance. 
When more than one bone has to treated simultaneously, prior 
knowledge of soft tissues becomes inevitable. The soft tissues 
are modeled using the geometric model of the individual 
tissue volume or using mechanical model that describes the 
deformation of the biological tissues. The bones and the soft 
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tissues lie on the same boundary and hence the tissue 
deformation can be computed from the displacement of 
boundaries. This system also integrates anisotropic and non 
linear elasticity properties of the tissues. 



Fig. 10. Bimaxillary osteotomy with soft tissue simulation 

This system as a whole thus provides vivid patient 
information for surgical training and education. 

G. VR application in Cardiac surgery 

To evaluate the congenital heart disease in children, 3D 
Echocardiographic Intracardiac Endoscopic Simulation 
System (3DE IESS) was introduced in [15]. This provides 
better understanding of intracardiac structures than 2D 
echocardiography. Since heart is in constant motion with 
complex anatomy, 2D imaging provides only partial 
information about the temporal and spatial relationship 
between structures. Similar to all other surgeries, the existing 
method of visualization of heart for surgery from 2D imaging 
modality is only through mental reconstruction. The authors 
of this paper have generated the 3DE datasets using a 2 to 4 
MHz matrix array transducer and stored in a CD-ROM for 
post processing and data analysis. The intra cardiac anatomy 
of the heart is examined using VR and any point inside the 
heart can be analysed with the help of virtual eye. The virtual 
eye can be rotated at any angle with the help of mouse and 
visualization of interior of the atrium and ventricle are also 
possible. The software for the echocardiographic analysis 
system has been written in Visual C++ 6.0 and graphics and 
visualization is performed using VTK (Visualization ToolKit, 
Kitware, Inc.). On experimental analysis on patients using 2D 
and 3D echocardiography, it was found that 3DE provides 
more reliable and better understanding of heart structures in 
cases with complex heart diseases. Inspite of its efficient 
visualization, the process is time consuming. Fig. 11 shows 
the virtual endoscopic view of a normal heart from the 
pulmonary artery. 




RPA 

I. PA 


Fig.l 1. Virtual endoscopic view of a normal heart by 3DE 
IESS, LPA-Left Pulmonary Artery, RPA-Right Pulmonary 

Artery 


III. ROLE OF VR IN POST ACCIDENT TRAUMA CARE 

A. VR Simulation in Abdominal trauma surgery 

Trauma is an emergency condition that cannot be taken risk 
of. Training the physicians to deal with traumatic injuries can 
be done by simulating the cases in virtual reality. One such 
simulation to remove a shattered kidney is presented in [16]. 
The simulator includes a special horizontally mounted 
computer system that provides the visualization of the 
patient's abdomen which is attached with head-like and legs- 
like dummies to give a realistic effect of a patient as shown in 
Fig. 12. 



Fig. 12. General setup of the simulator 

The patient records and the training details are provided in the 
GUI form by the software embedded with the simulator 
system. The software is composed of 6 modules each of 
which takes care of different simulator components. To 
achieve better efficiency, the software modules are 
maintained distributed in two systems - SGI ONYX-2 and 
SGI IMPACT which share memory. 

The deformations of the abdominal structures are modeled 
using standard mass-spring system. In order to maintain the 
default shape of the mass-spring surface, strut springs are 
used for anchoring. The real structures deform when any 
external stimulus like incisions occur. This can be achieved 
by a set of basic operations applied to individual triangles of 
the surface. The cutting can occur across any direction to any 
dimension and all the related deformities should reflect in all 
other abdominal parts. The arteries and tubular structures are 
modeled using simple linear mass-spring model. 



Fig. 13. Abdominal part after initial incision by user 

The blood flow during the accident and the incision is 
modeled using diffusion algorithm on the surface of 
polygonal models. The simulator system is incorporated with 
GHOST software package to provide collision detection and 
force feedback. With these availabilities, the simulation of 
shattered kidney removal can be well exposed and thus 
improve the physicians' expertise to deal with similar cases in 
the future. 

IV DISCUSSION 

This paper discussed about the technological advancements 
in VR and its tremendous support to the field of medicine. 
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Anatomy and abnormality is patient specific; surgical 
simulation and pre operative planning provides better 
visualization and interaction with patient’s inner anatomy. 
This helps training surgeons and interns in minimal invasive 
surgery rather than using cadavers, plastic models etc. 
Moreover, the skills learnt using simulators guide in reducing 
errors significantly in live surgery. The efficacy of the virtual 
environment depends on the freedom of interaction between 
the user and the simulated environment. With adequate 
infrastructure and training, VR emerges as a life saving tool 
in trauma management also. 
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